Predictive models based on various indicators of student engagement and/or regularity of study

Models based on ‘basic’ indicators of engagement and regularity of study

Loading the required data

daily.counts <- read.csv("Intermediate_results/regularity_of_study/weekly_counts_of_daily_logins_w2-13.csv")
#colnames(daily.counts)
weekly.counts <- daily.counts %>%
  select(user_id, W2_cnt:W13_cnt, tot_cnt, weekly_entropy)
# str(weekly.counts)
daily.gaps <- read.csv("Intermediate_results/regularity_of_study/gaps_between_consecutive_logins_w2-13.csv")
# str(daily.gaps)
# daily gaps do not have normal distribution, so, median will be used
# merge weekly counts and median time gap 
counts.data <- merge(x = weekly.counts, y = daily.gaps %>% select(user_id, median_gap),
                     by = 'user_id', all = TRUE)
exam.scores <- read.csv(file = "Intermediate_results/exam_scores_with_student_ids.csv")
# remove email data
exam.scores <- exam.scores %>% select(-2)
# str(exam.scores)
# merge counts data with exam scores
counts.data <- merge(x = counts.data, y = exam.scores, by.x = 'user_id', by.y = 'USER_ID', 
                     all.x = T, all.y = F)
#summary(counts.data)
# 9 NA values for exam scores; remove them
counts.data <- counts.data %>% filter( is.na(SC_FE_TOT)==FALSE )

Model 1: predictors are the same as those used in the latest cluster model

This means that predictors are counts of active days (days when a student had at least one learning session) per week, entropy of weekly active days, and median gap between two consecutive active days.

lm1.data <- counts.data %>% select(-c(tot_cnt, user_id, SC_MT_TOT))
lm1 <- lm(SC_FE_TOT ~ ., data = lm1.data)
summary(lm1)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm1.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-23.2397  -6.3099  -0.6918   5.9104  19.9677 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)      8.2257     2.3354   3.522  0.00047 ***
W2_cnt           0.4985     0.3431   1.453  0.14685    
W3_cnt           0.0926     0.3651   0.254  0.79991    
W4_cnt           0.1341     0.3532   0.380  0.70441    
W5_cnt          -0.3261     0.3228  -1.010  0.31290    
W6_cnt          -0.1632     0.3676  -0.444  0.65736    
W7_cnt           0.5343     0.4188   1.276  0.20266    
W8_cnt           1.0919     0.4068   2.685  0.00752 ** 
W9_cnt           0.1106     0.3862   0.286  0.77475    
W10_cnt          1.0810     0.3454   3.130  0.00186 ** 
W11_cnt          0.1618     0.3891   0.416  0.67768    
W12_cnt          0.4559     0.3298   1.383  0.16746    
W13_cnt          0.8561     0.2901   2.951  0.00333 ** 
weekly_entropy  11.6096     9.9681   1.165  0.24475    
median_gap      -0.2072     0.3777  -0.549  0.58350    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.417 on 462 degrees of freedom
Multiple R-squared:  0.2831,    Adjusted R-squared:  0.2614 
F-statistic: 13.03 on 14 and 462 DF,  p-value: < 2.2e-16

It’s interesting that counts for only 3 weeks are significant and that all three weeks are in the 2nd part of the course (after midterm exam):

• one day more of study in week 8 contributes 1.09 points to the final exam score;
• one day more in week 10 controbutes 1.08 points, and
• one active day more in week 13 adds 0.86 points.

R-squared is 0.283 (adjusted R2: 0.261).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm1$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm1)
par(mfrow=c(1,1)) # Change back to 1 x 1

# there are few potential influential points: 80, 50, 459
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:14)
  print(cor.test(lm1.data[,c], lm1$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm1)
# OK, values below or equal to 2

The assumptions are satisifed, though there are few potentially influential points that might need to be considered if this model is to be used

Model 2: Like Model 1, but instead of weekly counts, uses total number of active days during the course

lm2.data <- counts.data %>% select(tot_cnt, median_gap, weekly_entropy, SC_FE_TOT)
lm2 <- lm(SC_FE_TOT ~ ., data = lm2.data)
summary(lm2)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm2.data)

Residuals:
    Min      1Q  Median      3Q     Max 
-25.649  -6.003  -0.660   5.943  20.407 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)     5.84084    2.17939   2.680  0.00762 ** 
tot_cnt         0.40120    0.04530   8.857  < 2e-16 ***
median_gap     -0.05991    0.37409  -0.160  0.87284    
weekly_entropy 11.87966    9.90829   1.199  0.23114    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.476 on 473 degrees of freedom
Multiple R-squared:  0.2557,    Adjusted R-squared:  0.251 
F-statistic: 54.17 on 3 and 473 DF,  p-value: < 2.2e-16

The total number of active days is the only significant predictor, and it is highly significant. Each additional active day contributes 0.4 points to the final exam score.

R-squared is 0.2557 (adjusted R2: 0.251).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm2$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm2)
par(mfrow=c(1,1)) # Change back to 1 x 1

# both OK
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:3)
  print(cor.test(lm2.data[,c], lm2$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm2)
# OK, values below or equal to 2

All assumptions are satisified.

Model 3: Number of study sessions per week, weekly entropy of study session counts, and time gap between consecutive sessions

Loading the required data

weekly.sessions <- read.csv("Intermediate_results/regularity_of_study/weekly_session_props.csv")
#str(weekly.sessions)
ses.gap.data <- read.csv("Intermediate_results/regularity_of_study/inter-session_time_intervals.csv") #str(ses.gap.data)
lm3.data <- merge(x = weekly.sessions %>% select(count_w2:count_w12, weekly_entropy, user_id),
                  y = ses.gap.data %>% select(user_id, median_s_gap),
                  by = 'user_id', all = TRUE)
lm3.data <- merge(x = lm3.data, y = exam.scores %>% select(USER_ID, SC_FE_TOT),
                  by.x = 'user_id', by.y = 'USER_ID', all.x = T, all.y = F)
summary(lm3.data)
## remove rows with NAs
lm3.data <- lm3.data %>% filter(is.na(SC_FE_TOT)==FALSE & is.na(median_s_gap)==FALSE) 
# ## since some of the predictors are on quite different scales, rescale them
# apply(lm3.data %>% select(-user_id), 2, shapiro.test)
# apply(lm3.data %>% select(-user_id), 2, function(x) length(boxplot.stats(x)$out))
# # all preditors have outliers -> normalization is not advised
# lm3.sc.data <- scale.features(lm3.data %>% select(-c(user_id, SC_FE_TOT)))
# lm3.sc.data <- cbind(lm3.sc.data, SC_FE_TOT=lm3.data$SC_FE_TOT)
# summary(lm3.sc.data)
## the same results are obtained with scaled and unscaled (original) data;
## will keep the results with original data as they are easier to interpret
lm3.data <- lm3.data %>% select(-user_id)

lm3 <- lm(SC_FE_TOT ~ ., data = lm3.data)
summary(lm3)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm3.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-21.6292  -5.9597  -0.8594   5.9990  21.2523 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)    15.2754010  1.6511046   9.252  < 2e-16 ***
count_w2        0.0053628  0.1110213   0.048  0.96149    
count_w3        0.1909662  0.1157368   1.650  0.09962 .  
count_w4       -0.0832153  0.1005015  -0.828  0.40810    
count_w5       -0.2802013  0.0866320  -3.234  0.00131 ** 
count_w7        0.2530987  0.1498286   1.689  0.09184 .  
count_w8        0.1688238  0.1441518   1.171  0.24214    
count_w9        0.1951703  0.1603609   1.217  0.22420    
count_w10       0.5158746  0.1080020   4.777  2.4e-06 ***
count_w11       0.3756283  0.1545471   2.431  0.01546 *  
count_w12       0.1057666  0.1333451   0.793  0.42808    
weekly_entropy 25.4435546  8.4577490   3.008  0.00277 ** 
median_s_gap   -0.0001634  0.0003508  -0.466  0.64167    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.31 on 463 degrees of freedom
Multiple R-squared:  0.2938,    Adjusted R-squared:  0.2755 
F-statistic: 16.05 on 12 and 463 DF,  p-value: < 2.2e-16

Significant predictors:

• session counts in week 5: negative impact (!), an additional session decreases exam score by 0.28
• session counts in week 10: an additional session increases exam score by 0.52
• session counts in week 11: an additional session increases exam score by 0.37
• weekly entropy: if entropy increases by 1, exam score increases by 25; however, entropy cannot increase not nearly that much; this simply confirms that the higher the regularity (high entropy means that weekly counts are almost uniformly distributed), the higher the performance

R-squared: 0.294 (adjusted R2: 0.275).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm3$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm3)
par(mfrow=c(1,1)) # Change back to 1 x 1

# mostly fine, but there are few (potentially) influential points: 412, 459, 437, 77
# let's examine them
lm3.data[c(412,459,437,77),]

summary(lm3.data)
# 437 has very low engagement and very high exam score (35)
# 459 has high engagement (at times very high) and zero (0) exam score
# 412 is similar to 459, but not that extreme (7 exam score; less active)
# 77 is almost completely inactive, and has zero (0) exam score
## assumption 4: predictors and residuals are uncorrelated
lm3.data <- lm3.data %>% filter(is.na(median_s_gap)==FALSE)
for(c in 1:12)
  print(cor.test(lm3.data[,c], lm3$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm3)
# OK, values below or slightly above 2

Model 4: Total number of study sessions, weekly entropy of study session counts, and time gap between consecutive sessions

lm4.data <- merge(x = weekly.sessions %>% select(user_id, s_total, weekly_entropy),
                  y = ses.gap.data %>% select(-mad_s_gap),
                  by = 'user_id', all = TRUE)
lm4.data <- merge(x = lm4.data, y = exam.scores %>% select(USER_ID, SC_FE_TOT),
                  by.x = 'user_id', by.y = 'USER_ID', all.x = T, all.y = F)
lm4.data <- lm4.data %>% filter( is.na(SC_FE_TOT)==FALSE ) %>% select(-user_id)

lm4 <- lm(SC_FE_TOT ~ ., data = lm4.data)
summary(lm4)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm4.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-29.4102  -6.3591  -0.7392   6.5142  19.4481 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)    1.526e+01  1.658e+00   9.205  < 2e-16 ***
s_total        1.235e-01  1.383e-02   8.926  < 2e-16 ***
weekly_entropy 3.188e+01  8.598e+00   3.708 0.000233 ***
median_s_gap   2.975e-05  3.607e-04   0.082 0.934310    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.599 on 472 degrees of freedom
  (1 observation deleted due to missingness)
Multiple R-squared:  0.2292,    Adjusted R-squared:  0.2243 
F-statistic: 46.78 on 3 and 472 DF,  p-value: < 2.2e-16

Significant predictors:

• total number of sessions: an additional session increases exam score by 0.12
• weekly entropy: if entropy increases by 1, exam score increases by 31.88; however, entropy cannot increase not nearly that much; this simply confirms that the higher the regularity (high entropy means that weekly counts are almost uniformly distributed), the higher the performance

R-squared: 0.229 (adjusted R2: 0.224).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm4$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm4)
par(mfrow=c(1, 1))

# the Residuals vs Fitted plot suggests that there might be some non-linear realtionship between the outcome and the predictors
# there are also few influential points: 412, 459, 376, 77
# let's examine them
lm4.data[c(412,459,376, 77),]

summary(lm4.data)
# 77 is a clear outlier
# 376 has relatively high engagement (above 3rd quartile), but very low exam score (4)
# 412 and 459 have already been examined before
## assumption 4: predictors and residuals are uncorrelated
lm4.data <- lm4.data %>% filter(is.na(median_s_gap)==FALSE)
for(c in 1:3)
  print(cor.test(lm4.data[,c], lm4$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm4)
# OK, values below 2

Model 5: Number of study sessions per week day, and week day entropy of study session counts

Loading the data

weekday.sessions <- read.csv("Intermediate_results/regularity_of_study/weekday_session_props.csv")
#str(weekday.sessions)
lm5.data <- merge(x = weekday.sessions %>% select(1:8, 11),
                  y = exam.scores %>% select(-SC_MT_TOT),
                  by.x = "user_id", by.y = "USER_ID",
                  all.x = TRUE, all.y = FALSE)
# summary(lm5.data)
lm5.data <- lm5.data %>% filter( is.na(SC_FE_TOT)==FALSE ) %>% select(-user_id)

lm5 <- lm(SC_FE_TOT ~ ., data = lm5.data)
summary(lm5)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm5.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-31.2357  -6.1439  -0.9892   6.7715  20.2779 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)     15.07048    2.26948   6.641 8.69e-11 ***
Sun_count        0.08458    0.09785   0.864 0.387818    
Mon_count        0.20397    0.04877   4.182 3.44e-05 ***
Tue_count        0.11681    0.03721   3.140 0.001799 ** 
Wed_count        0.10067    0.04187   2.405 0.016578 *  
Thu_count        0.16141    0.04133   3.905 0.000108 ***
Fri_count        0.12135    0.10226   1.187 0.235970    
Sat_count       -0.02262    0.12568  -0.180 0.857269    
weekday_entropy 17.61256    6.86108   2.567 0.010567 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.663 on 468 degrees of freedom
Multiple R-squared:  0.2307,    Adjusted R-squared:  0.2176 
F-statistic: 17.55 on 8 and 468 DF,  p-value: < 2.2e-16

Significant predictors:

• Monday session counts: an additional Mon session increases exam score by 0.204
• Tuesday session counts: an additional Tue session increases exam score by 0.117
• Wednesday session counts: an additional Wed session increases exam score by 0.1
• Thursday session counts: an additional Thu session increases exam score by 0.161
• weekly entropy: if entropy increases by 1, exam score increases by 17.6; however, entropy cannot increase not nearly that much; this simply confirms that the higher the regularity (high entropy means that weekly counts are almost uniformly distributed), the higher the performance

R-squared: 0.231 (adjusted R2: 0.218).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm5$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm5)
par(mfrow=c(1,1)) # Change back to 1 x 1

# mostly fine, but there are few potentially influential points: usual suspects (412, 459), 230
# let's examine them
lm5.data[c(412,459,230),]

summary(lm5.data)
# 230 has low engagement and very high exam score (35)
# 459 and 412 have already been considered
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:8)
  print(cor.test(lm5.data[,c], lm5$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm5)
# OK, values below 2

Models based on ‘advanced’ indicators of engagement and regularity of study

Model 6: Daily resource use

As predictors, use total counts of different kinds of resources students used during their active days (an active day is a day when a student had at least one study session). The types of resources considered:

• video (VIDEO)
• exercises (EXE)
• multiple choice questions (MCQ)
• reading materials (RES)
• metacognitive items (METACOG)

In addition, consider using:

• median_X_cnt - median number of resources of type X used during the student’s active days (X can be VIDEO, EXE, MCQ, RES, METACOG)
• mad_X_cnt - MAD (median absolute deviation) of resources of the type X used during the student’s active days
• days_X_used - number of days when resources of the type X were used
• prop_X_used - proportion of days when resources of the type X were used versus total number of the student’s active days

Loading the data…

res.use.stats <- read.csv("Intermediate_results/regularity_of_study/daily_resource_use_statistics_w2-5_7-12.csv")
#str(res.use.stats)
lm6.data <- merge(res.use.stats, exam.scores, by.x = "user_id", by.y = "USER_ID", all.x = T, all.y = F)
lm6.data <- lm6.data %>% select(-c(user_id, SC_MT_TOT)) %>% filter( is.na(SC_FE_TOT)==FALSE )

First, include only the indicators of engagement (not regularity)

lm6_1.data <- lm6.data %>% select( starts_with("tot"), starts_with("prop"), SC_FE_TOT)
# examine the presence of (high) correlation between the variables
ggcorr(lm6_1.data, method = c("complete","spearman"), 
       #      geom = "circle", min_size = 0, max_size = 15,
       label = TRUE, label_size = 3.5,
       hjust = 0.85, size = 4, layout.exp = 1)

# tot_mcog_cnt and prop_mcog_used are highly correlated, as are tot_video_cnt and prop_video_used, and tot_mcq_cnt and prop_mcq_used 
lm6_1.data <- lm6_1.data %>% select(-c(prop_mcog_used, prop_video_used, prop_mcq_used))

# remove the outliers and re-run the model
lm6_1.data <- lm6_1.data[-c(86, 412, 462, 459),]
lm6_1 <- lm(SC_FE_TOT ~., data = lm6_1.data)
summary(lm6_1)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm6_1.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-22.0996  -5.9578  -0.1087   6.5627  20.6982 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)    6.505e+01  2.572e+01   2.529   0.0118 *  
tot_video_cnt  1.314e-03  8.603e-04   1.528   0.1273    
tot_exe_cnt   -6.820e-03  1.328e-03  -5.135 4.16e-07 ***
tot_mcq_cnt    8.755e-03  3.661e-03   2.392   0.0172 *  
tot_mcog_cnt   9.811e-03  1.281e-02   0.766   0.4441    
tot_res_cnt    1.008e-02  1.955e-03   5.156 3.74e-07 ***
prop_exe_used  4.138e-01  3.661e+00   0.113   0.9100    
prop_res_used -4.626e+01  2.576e+01  -1.796   0.0732 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.798 on 465 degrees of freedom
Multiple R-squared:  0.1922,    Adjusted R-squared:  0.1801 
F-statistic: 15.81 on 7 and 465 DF,  p-value: < 2.2e-16

Significant predictors:

• total exercise counts (number of exercise-related events during the student’s active days): an additional exercise-related event decreases the final exam score by 0.0066
• total MCQ counts (number of MCQ-related events during the student’s active days): an additional MCQ-related event increases the final exam score by 0.0078
• total number of reading related events: an additional reading-related event decreases the final exam score by 0.0096

R-squared: 0.192 (adjusted R2: 0.180).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm6_1$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm6_1)
par(mfrow=c(1, 1))

# unclear if homoscedasticity requirement is fulfilled; check using this plot:
check.homoschedasticity(lm6_1)

# not that good
# # the plots point to couple of outliers: 86, 412, 462, 459 
# # let's check them:
# lm6_1.data[c(86, 412, 462, 459),]
# summary(lm6_1.data)
# # 459 and 462 have zero exam score, inspite of non-negligible number of learning events (especially 459)
# # 412 was highly active, but had very low exam score (7)
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:7)
  print(cor.test(lm6_1.data[,c], lm6_1$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm6_1)
# it's fine: all below or equal to 2

The assumption of homoscedasticity cannot be considered satisfied (even after removing outliers)

Now, include both indicators of engagement and indicator of regularity

# include those engagment indicators that proved at least slightly relevant in the previous model
# plus mad_X_cnt as indicators of regularity
lm6_2.data <- lm6.data %>% select(tot_mcq_cnt, tot_exe_cnt, tot_res_cnt, prop_res_used, 
                                  starts_with("mad"), SC_FE_TOT)
# examine the presence of (high) correlation between the variables
plot.correlations(lm6_2.data)

# exclude mad_res_cnt as highly correlated with tot_res_cnt (which proved significant)
lm6_2.data <- lm6_2.data %>% select(-mad_res_cnt)

lm6_2 <- lm(SC_FE_TOT ~., data = lm6_2.data)
summary(lm6_2)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm6_2.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-22.9335  -6.1144  -0.1174   6.8471  23.1086 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)    66.127636  26.170686   2.527 0.011840 *  
tot_mcq_cnt     0.012136   0.003640   3.334 0.000923 ***
tot_exe_cnt    -0.006543   0.001360  -4.813 2.01e-06 ***
tot_res_cnt     0.010416   0.002000   5.208 2.87e-07 ***
prop_res_used -47.054649  26.296683  -1.789 0.074201 .  
mad_video_cnt  -0.007622   0.081861  -0.093 0.925861    
mad_exe_cnt    -0.003117   0.020326  -0.153 0.878197    
mad_mcq_cnt    -0.543694   0.379258  -1.434 0.152362    
mad_mcog_cnt   -0.952299   1.089663  -0.874 0.382599    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.983 on 468 degrees of freedom
Multiple R-squared:  0.1728,    Adjusted R-squared:  0.1586 
F-statistic: 12.22 on 8 and 468 DF,  p-value: 6.383e-16

None of the MAD variables is significant

Model 7: Daily topic focus

As predictors, use total number of learning actions (during active days) with a particular topic focus; possible topic foci:

• ‘ontopic’ - the topic associated with the action is the topic of the current week
• ‘revisiting’ - the topic associated with the action is the topic of one of the previous weeks
• ‘metacognitive’ - the topic associated with the action is one of the following: ‘ORG’, ‘DBOARD’, ‘STRAT’, ‘STUDYKIT’
• ‘orienteering’ - the topic associated with the action is one of the following: ‘HOME’, ‘HOF’, ‘SEARCH’, ‘TEA’, ‘EXAM’, ‘W01’-‘W13’
• ‘project’ - the action is related to project work

In addition, consider including the following basic statistics:

• median_X_cnt - median number of learning actions per active day with a particular topic focus
• mad_X_cnt - MAD of learning actions per active day with a particular topic focus
• X_days - number of days with at least one action with particular topic focus
• X_prop - proportion of days with the given type of topic focus versus total number of active days

Loading the required data…

topic.stats <- read.csv("Intermediate_results/regularity_of_study/topic_counts_statistics_w2-5_7-12.csv")
# str(topic.stats)
lm7.data <- merge(topic.stats, exam.scores, by.x = "user_id", by.y = "USER_ID", all.x = T, all.y = F)
lm7.data <- lm7.data %>% select(-c(user_id, SC_MT_TOT)) %>% filter( is.na(SC_FE_TOT)==FALSE )

First, include only the indicators of engagement (not regularity)

lm7_1.data <- lm7.data %>% select( starts_with("tot"), ends_with("prop"), SC_FE_TOT)
summary(lm7_1.data)
# examine the presence of (high) correlation between the variables
plot.correlations(lm7_1.data)

# exclude tot_orient_cnt and orinet_prop as they are highly correlated with some other variables
lm7_1.data <- lm7_1.data %>% select(-c(tot_orient_cnt, orient_prop))

# exclude tot_prj_cnt, due to high VIF
lm7_1.data <- lm7_1.data %>% select(-tot_prj_cnt)
lm7_1 <- lm(SC_FE_TOT ~ ., data = lm7_1.data)
summary(lm7_1)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm7_1.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-22.4947  -6.5546  -0.6803   6.5096  21.4971 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)      4.592e+01  1.040e+01   4.413 1.27e-05 ***
tot_ontopic_cnt  4.854e-03  9.985e-04   4.861 1.60e-06 ***
tot_revisit_cnt -5.237e-03  1.455e-03  -3.600 0.000352 ***
tot_metacog_cnt  1.191e-02  3.565e-03   3.341 0.000901 ***
ontopic_prop     8.082e-01  3.322e+00   0.243 0.807890    
revisit_prop     1.940e+00  3.106e+00   0.625 0.532537    
metacog_prop    -3.469e+01  1.017e+01  -3.410 0.000705 ***
prj_prop         4.500e+00  4.893e+00   0.920 0.358177    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9.063 on 469 degrees of freedom
Multiple R-squared:  0.1562,    Adjusted R-squared:  0.1436 
F-statistic:  12.4 on 7 and 469 DF,  p-value: 1.347e-14

Significant predictors:

• total number of prepartion (ontopic) events: an additional ontopic event increases the final exam score by 0.0048
• total number of revisiting events: an additional revisting event decreases the final exam score by 0.0052
• total number of metacognitive events: an additional metacognitive event increases the final exam score by 0.012
• proportion of days with metacognitive events versus total number of active days: increase in this proportion decreases the exam score

R-squared: 0.156 (adjusted R2: 0.145).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm7_1$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm7_1)
par(mfrow=c(1, 1))

# normality is fine
# unclear if homoscedasticity requirement is fulfilled; check using this plot:
check.homoschedasticity(lm7_1)

# it's fine
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:8)
  print(cor.test(lm7_1.data[,c], lm7_1$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm7_1)
# now, it's fine

Now, include both indicators of engagement and indicator of regularity

# include those engagment indicators that proved at least slightly relevant in the previous model
# plus mad_X_cnt as indicators of regularity
lm7_2.data <- lm7.data %>% select(tot_ontopic_cnt, tot_revisit_cnt, tot_metacog_cnt, metacog_prop, 
                                  starts_with("mad"), SC_FE_TOT)
# examine the presence of (high) correlation between the variables
plot.correlations(lm7_2.data)

# exclude tot_metacog_cnt as highly correlated with mad_metacog_cnt and mad_orient_cnt
lm7_2.data <- lm7_2.data %>% select(-c(tot_metacog_cnt, mad_orient_cnt))

lm7_2 <- lm(SC_FE_TOT ~., data = lm7_2.data)
summary(lm7_2)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm7_2.data)

Residuals:
    Min      1Q  Median      3Q     Max 
-20.586  -6.413  -1.047   6.841  22.230 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)      4.254e+01  9.826e+00   4.330 1.83e-05 ***
tot_ontopic_cnt  7.177e-03  9.323e-04   7.698 8.27e-14 ***
tot_revisit_cnt -3.802e-03  1.352e-03  -2.812  0.00513 ** 
metacog_prop    -2.679e+01  1.019e+01  -2.629  0.00885 ** 
mad_ontopic_cnt -1.257e-01  4.010e-02  -3.135  0.00182 ** 
mad_revisit_cnt -9.064e-02  7.364e-02  -1.231  0.21897    
mad_metacog_cnt -3.913e-02  1.476e-01  -0.265  0.79102    
mad_prj_cnt     -7.659e-01  4.124e+00  -0.186  0.85276    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9.1 on 469 degrees of freedom
Multiple R-squared:  0.1493,    Adjusted R-squared:  0.1366 
F-statistic: 11.76 on 7 and 469 DF,  p-value: 8.013e-14

The only regularity indicator that proved significant: mad_ontopic_cnt - one unit increase in MAD of ontopic counts leads to a decrease of 0.126 points in the final exam score

R2: 0.1493 (adjusted R2: 0.1366).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm7_2$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm7_2)
par(mfrow=c(1, 1))

# normality is fine
# unclear if homoscedasticity requirement is fulfilled; check using this plot:
check.homoschedasticity(lm7_2)

# not bad
# a few influential points: 60, 54, 202
# and a few outliers: 19, 294, 50
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:7)
  print(cor.test(lm7_2.data[,c], lm7_2$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm7_2)
# OK

A few outliers and (potentially) influential points; apart from that, it’s fine

Model 8: Weekly resource use indicators

Indicators are computed at the week level, based on the following principle: a score of one is given to a student (for a given week), if he/she used certain kind of resource (e.g. video) more than the average (median) use of the that resource type in the given week

Loading the data…

res.use.ind <- read.csv("Intermediate_results/regularity_of_study/res_use_indicators_w2-13.csv")
#str(res.use.ind)
lm8.data <- merge(x = res.use.ind, y = exam.scores %>% select(USER_ID, SC_FE_TOT),
                  by.x = "user_id", by.y = "USER_ID", all.x = TRUE, all.y = FALSE)
#summary(lm8.data)
# remove students who do not have final exam score
lm8.data <- lm8.data %>% filter( is.na(SC_FE_TOT)==FALSE )
lm8.data <- lm8.data %>% select(-user_id)
# examime correlations
plot.correlations(lm8.data)

# video_ind and MCQ_ind are highly correlated, remove one of them
lm8.data <- lm8.data %>% select(-VIDEO_ind)

lm8 <- lm(SC_FE_TOT ~ ., data = lm8.data)
summary(lm8)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm8.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-22.7333  -5.8524  -0.1573   6.1593  21.4361 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 16.10045    1.07114  15.031  < 2e-16 ***
MCQ_ind      0.95088    0.16395   5.800 1.22e-08 ***
EXE_ind     -0.82529    0.14298  -5.772 1.42e-08 ***
RES_ind      0.60658    0.14833   4.089 5.08e-05 ***
METACOG_ind  0.03796    0.15725   0.241    0.809    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.38 on 472 degrees of freedom
Multiple R-squared:  0.2739,    Adjusted R-squared:  0.2678 
F-statistic: 44.52 on 4 and 472 DF,  p-value: < 2.2e-16

Significant predictors:

• MCQ_ind - a unit increase in this indicator (ie, one week more when a student’s use of MCQs is higher than the average (median) use of MCQ in that week), increases the final exam score by 0.951 points
• EXE_ind - a unit increase of this indicator (ie, one week more when a student’s use of exercises is higher than the average (median) use of exercises in that week), decreases the exam score by 0.825 points
• RES_ind - a unit increase of this indicator (ie, one week more when a student’s use of reading materias is higher than the average (median) use of reading content in that week), increases the exam score by 0.606 points.

R-squared: 0.274 (adjusted R2: 0.268).

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm8$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm8)
par(mfrow=c(1, 1))

# both normality and homoscedasticity requirements are fulfilled
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:4)
  print(cor.test(lm8.data[,c], lm8$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm8)
# it's fine

Model 9: Topic focus indicators

Indicators are computed at the week level, based on the following principle: a score of one is given to a student (for a given week), if his/her number of events related to a particular topic type (e.g. revisiting) was above the average (median) number of events with that topic type in the given week

Weeks 6 and 13 are excluded from these computations, as during these weeks one can expect different behavioral patterns than usual.

Loading the data

topic.ind <- read.csv("Intermediate_results/regularity_of_study/topic_based_indicators_w2-5_7-12.csv")
#str(topic.ind)
lm9.data <- merge(x = topic.ind, y = exam.scores %>% select(USER_ID, SC_FE_TOT),
                  by.x = "user_id", by.y = "USER_ID", all.x = TRUE, all.y = FALSE)
#summary(lm9.data)
# remove students who do not have final exam score
lm9.data <- lm9.data %>% filter( is.na(SC_FE_TOT)==FALSE )
lm9.data <- lm9.data %>% select(-user_id)
plot.correlations(lm9.data)

# orient_ind and metacog_ind are highly correlated, remove one of them
lm9.data <- lm9.data %>% select(-orient_ind)

lm9 <- lm(SC_FE_TOT ~ ., data = lm9.data)
summary(lm9)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm9.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-21.7208  -6.8879  -0.8953   7.0348  23.7086 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  13.2952     1.0797  12.314  < 2e-16 ***
ontopic_ind   0.9303     0.1693   5.496 6.38e-08 ***
revisit_ind  -0.2917     0.1714  -1.702   0.0893 .  
metacog_ind   0.4606     0.2082   2.212   0.0274 *  
prj_ind       0.2421     0.3127   0.774   0.4392    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9.076 on 472 degrees of freedom
Multiple R-squared:  0.1484,    Adjusted R-squared:  0.1412 
F-statistic: 20.56 on 4 and 472 DF,  p-value: 1.236e-15

Significant predictors:

• ontopic_ind - a unit increase in this indicator (ie, one week more when a student’s number of ‘ontopic’ events is higher than the average (median) number of ‘ontopic’ events in that week), increases the final exam score by 0.9303 points
• metacog_ind - a unit increase in this indicator (ie, one week more when a student’s number of ‘metacognitive’ events is higher than the average (median) number of ‘metacognitive’ events in that week), increases the final exam score by 0.4606 points

R-squared: 0.1484 (adjusted R-squared: 0.1412)

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm9$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm9)
par(mfrow=c(1, 1))

# both normality and homoscedasticity requirements are fulfilled
# there are few outliers (202, 213, 365), but no influential points
# check the outliers
lm9.data[c(202,213,365),]

# very interesting:
# - 202 was highly active but ended up with zero final exam score
# - 213 and 365 were not preparing for lectures, and generaly had low engagement, but did the exam excellently
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:4)
  print(cor.test(lm9.data[,c], lm9$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm9)
# it's fine

Model 10: Combine resource use and topic focus indicators

Use those indicators that proved significant in the previous two models (models 8 and 9)

Loading the data

topic.ind <- read.csv("Intermediate_results/regularity_of_study/topic_based_indicators_w2-5_7-12.csv")
res.use.ind <- read.csv("Intermediate_results/regularity_of_study/res_use_indicators_w2-13.csv")
lm10.data <- merge(x = topic.ind %>% select(user_id, ontopic_ind, metacog_ind), 
                   y = res.use.ind %>% select(user_id, MCQ_ind, RES_ind, EXE_ind),
                  by = "user_id", all = TRUE)
lm10.data <- merge(x = lm10.data, y = exam.scores %>% select(USER_ID, SC_FE_TOT),
                  by.x = "user_id", by.y = "USER_ID", all.x = TRUE, all.y = FALSE)
summary(lm10.data)
# remove students who do not have final exam score
lm10.data <- lm10.data %>% filter( is.na(SC_FE_TOT)==FALSE )
plot.correlations(lm10.data %>% select(-user_id))

# RES_ind and metacog_ind are highly correlated, remove metacog_ind as it was less significant in the previous models
lm10.data <- lm10.data %>% select(-metacog_ind)

lm10 <- lm(SC_FE_TOT ~ ., data = lm10.data %>% select(-user_id))
summary(lm10)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm10.data %>% select(-user_id))

Residuals:
     Min       1Q   Median       3Q      Max 
-21.6264  -5.5145  -0.4311   6.2478  22.6760 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  16.0064     1.0433  15.343  < 2e-16 ***
ontopic_ind   0.4506     0.2055   2.192 0.028843 *  
MCQ_ind       0.7235     0.1851   3.908 0.000107 ***
RES_ind       0.5582     0.1492   3.742 0.000205 ***
EXE_ind      -0.9275     0.1496  -6.199 1.24e-09 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.339 on 472 degrees of freedom
Multiple R-squared:  0.2812,    Adjusted R-squared:  0.2751 
F-statistic: 46.15 on 4 and 472 DF,  p-value: < 2.2e-16

All 4 predictors are significant; however, only slight improvement in R2 w.r.t. model 8: R-squared: 0.2812 (adjusted R-squared: 0.2751)

Model 11: Extend model 10 with total number of study sessions and entropy of weekly study session counts

Loading the data

weekly.sessions <- read.csv("Intermediate_results/regularity_of_study/weekly_session_props.csv")
lm11.data <- merge(x = lm10.data, y = weekly.sessions %>% select(user_id, s_total, weekly_entropy),
                   by = 'user_id', all.x = TRUE, all.y = FALSE)
#summary(lm11.data)
lm11.data <- lm11.data[,c(1:5,7,8,6)]
plot.correlations(lm11.data %>% select(-user_id))

# total number of sessions is highly correlated with almost all other variables
lm11.data <- lm11.data %>% select(-s_total)

lm11 <- lm(SC_FE_TOT ~ ., data = lm11.data %>% select(-c(user_id, ontopic_ind)))
summary(lm11)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm11.data %>% select(-c(user_id, 
    ontopic_ind)))

Residuals:
     Min       1Q   Median       3Q      Max 
-24.1085  -5.7500  -0.1444   5.9306  20.1971 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)     23.6142     1.5988  14.770  < 2e-16 ***
MCQ_ind          0.8916     0.1438   6.200 1.23e-09 ***
RES_ind          0.4795     0.1444   3.320 0.000969 ***
EXE_ind         -0.9923     0.1406  -7.060 6.00e-12 ***
weekly_entropy  42.6242     7.0907   6.011 3.69e-09 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.077 on 472 degrees of freedom
Multiple R-squared:  0.3255,    Adjusted R-squared:  0.3198 
F-statistic: 56.94 on 4 and 472 DF,  p-value: < 2.2e-16

All 4 predictors that were eventually used for model building - MCQ_ind, EXE_ind, RES_ind, and weekly_entropy - proved highly significant.

R-squared: 0.3255 (adjusted R-squared: 0.3198)

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm11$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm11)
par(mfrow=c(1, 1))

# normality is fulfilled, but the homoscedasticity requirements is questionable
check.homoschedasticity(lm11)

# not good, there outliers and/or influential points
# check the outliers
lm11.data[c(49,294,50),]

# - 294 and 50: low to moderate activity indicators and zero final exam score
# - 230: moderate activity indicators, but excellent exam score
# check influential points
inf.indices <- as.numeric(names(head(sort(cooks.distance(lm11), decreasing = T))))
lm11.data[inf.indices,]

# observations with ordinal numbers  163, 241, 336 should be considered for removal
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:5)
  print(cor.test(lm11.data[,c], lm11$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm11)
## ontopic_ind and MCQ_ind have values > 2; remove ontopic_ind as it is not significant
# now (after ontopic_ind was removed), it's fine

If the model is to be used, the outliers should be dealt with.

Model 12: Extend model 10 with number of study sessions per weekday, and weekday entropy of study session counts

In addition to predictors from Model 10 and weekday entropy of study session counts, use, as predictors, study session counts for those week days that proved as significant predictors in Model 5 (Mon, Tue, Wed, Thu).

Loading the data…

weekday.sessions <- read.csv("Intermediate_results/regularity_of_study/weekday_session_props.csv")
lm12.data <- merge(x = lm10.data,
                   y = weekday.sessions %>% select(user_id, Mon_count, Tue_count, Wed_count,
                                                   Thu_count, weekday_entropy), 
                   by = "user_id", all.x = TRUE, all.y = FALSE)
# summary(lm12.data)
lm12.data <- lm12.data[,c(1:5,7:11,6)]
plot.correlations(lm12.data %>% select(-user_id))

lm12 <- lm(SC_FE_TOT ~ ., data = lm12.data %>% select(-c(user_id, ontopic_ind, RES_ind)))
summary(lm12)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm12.data %>% select(-c(user_id, 
    ontopic_ind, RES_ind)))

Residuals:
     Min       1Q   Median       3Q      Max 
-26.9153  -5.6781  -0.2458   5.9728  21.6389 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)     22.93612    2.25082  10.190  < 2e-16 ***
MCQ_ind          0.59566    0.15805   3.769 0.000185 ***
EXE_ind         -1.00543    0.14347  -7.008 8.45e-12 ***
Mon_count        0.14117    0.04520   3.123 0.001901 ** 
Tue_count        0.11040    0.03598   3.068 0.002278 ** 
Wed_count        0.08320    0.04090   2.034 0.042474 *  
Thu_count        0.12923    0.04152   3.112 0.001970 ** 
weekday_entropy 26.94642    6.01675   4.479 9.45e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.085 on 469 degrees of freedom
Multiple R-squared:  0.3285,    Adjusted R-squared:  0.3185 
F-statistic: 32.78 on 7 and 469 DF,  p-value: < 2.2e-16

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm12$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm12)
par(mfrow=c(1, 1))

# both normality and homoscedasticity requirements are fulfilled
# there are few outliers (230, 50, 459), but they do not look that significant
# check influential points
inf.indices <- as.numeric(names(head(sort(cooks.distance(lm12), decreasing = T))))
lm12.data[inf.indices,]

# observations with ordinal numbers  459, 22, 292 should be considered for removal
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:10)
  print(cor.test(lm12.data[,c], lm12$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm12)
## ontopic_ind has value > 2; remove it
## now RES_ind stands out with high value; remove it
# now (after ontopic_ind and RES_ind were removed), it's fine

Model 13: Extends model 12 with entropy of weekly study session counts

Loading the data…

weekly.sessions <- read.csv("Intermediate_results/regularity_of_study/weekly_session_props.csv")
lm13.data <- merge(x = lm12.data %>% select(-c(ontopic_ind, RES_ind)), 
                   y = weekly.sessions %>% select(user_id, weekly_entropy),
                   by = 'user_id', all.x = TRUE, all.y = FALSE)
#summary(lm13.data)
lm13.data <- lm13.data[,c(1:8,10,9)]
plot.correlations(lm13.data %>% select(-user_id))

lm13 <- lm(SC_FE_TOT ~ ., data = lm13.data %>% select(-user_id))
summary(lm13)

Call:
lm(formula = SC_FE_TOT ~ ., data = lm13.data %>% select(-user_id))

Residuals:
     Min       1Q   Median       3Q      Max 
-25.4716  -5.6469  -0.3401   5.5829  20.8140 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)     25.49497    2.31501  11.013  < 2e-16 ***
MCQ_ind          0.64099    0.15619   4.104 4.79e-05 ***
EXE_ind         -1.05966    0.14208  -7.458 4.29e-13 ***
Mon_count        0.12474    0.04475   2.788 0.005526 ** 
Tue_count        0.08943    0.03587   2.493 0.013013 *  
Wed_count        0.07164    0.04041   1.773 0.076912 .  
Thu_count        0.10852    0.04127   2.630 0.008832 ** 
weekday_entropy 17.59079    6.40524   2.746 0.006259 ** 
weekly_entropy  29.83472    7.72548   3.862 0.000128 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 7.968 on 468 degrees of freedom
Multiple R-squared:  0.3492,    Adjusted R-squared:  0.3381 
F-statistic: 31.39 on 8 and 468 DF,  p-value: < 2.2e-16

R-squared: 0.349 (adjusted R-squared: 0.338)

Checking if the model satisfies the assumptions for linear regression:

# assumption 1: the mean of residuals is zero
mean(lm13$residuals)
# OK
# assumption 2: homoscedasticity of residuals or equal variance
# assumption 3: Normality of residuals
par(mfrow=c(2, 2))
plot(lm13)
par(mfrow=c(1, 1))

check.homoschedasticity(lm13)

# normality is fulfilled
# homoscedasticity is somewhat questionable - there are outliers and/or influential points
# outliers: 230, 50, 459
# check influential points
inf.indices <- head(sort(cooks.distance(lm13), decreasing = T))
inf.indices
lm13.data[as.numeric(names(inf.indices)),]

# observations with ordinal numbers  459, 163, 336, and 241 should be considered for removal (all have final exam score = zero)
## assumption 4: predictors and residuals are uncorrelated
for(c in 1:10)
  print(cor.test(lm13.data[,c], lm13$residuals))
# OK
## assumption 6: no multicolinearity between explanatory variables
vif(lm13)
# it's fine